PUMA: Performance Unchanged Model Augmentation for Training Data Removal

Preserving the performance of a trained model while removing unique characteristics marked training data points is challenging. Recent research usually suggests retraining from scratch with remaining or refining by reverting optimization on points. Unfortunately, aside their computational inefficiency, those approaches inevitably hurt resulting model's generalization ability since they remove not only but also discard shared (and possibly contributive) information. To address degradation problem, this paper presents novel approach called Performance Unchanged Model Augmentation (PUMA). The proposed PUMA framework explicitly models influence each point respect to various criteria. It then complements negative impact reweighting optimally. demonstrate effectiveness framework, we compared it multiple state-of-the-art removal techniques in experiments, where show can effectively and efficiently without that 1) fool membership attack, 2) resist degradation. In addition, as estimates importance during its operation, could serve debug mislabelled more than existing approaches.